IMAGE PROCESSING METHOD, IMAGE PROCESSOR, AND PROGRAM FOR CAUSING 
A COMPUTER TO EXECUTE THE PROCESSING METHOD 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to an image processing 
method and image processor that perform image processing on a 
synthesized image obtained by synthesizing a natural image 
acquired by photography and a computer graphic (CG) image such 
10 as clip art contained in a template, and to a program for causing 

a computer to execute the image processing method. 

Description of the Related Art 
In photo studios, there is service for making, for 
esample. New Year's cards illustrated with photographs, by 
15 utilizing a template that consists of New Year's characters, 

illustrations, and a blank region for inserting photographs. 
Such New Year's cards are made by, for example, synthesizing 
image data obtained by reading out an image recorded on a negative 
film or obtained with a digital camera, and template data 
20 representing a template containing clip art or characters 

produced by computer processing, and then printing a synthesized 
image on the basis of the synthesized image data. 

Besides photo studios, a synthesized image can also 
be made by synthesizing an image obtained by a user and a template 
25 by an application program for synthesizing and editing images 

installed in personal computers . This enables users to make post 
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cards easily. 

To enhance the picture quality of an image reproduced 
by printing out image data, an image-processing parameter for 
image processing, such as a density correction process, a color 
5 correction process, a gradation correction process, etc., is 

computed based on image data, and based on the computed 
image-processing parameter, image processing is performed on 
the image data. The image processing renders it possible to 
obtain a reproduced image with a high picture quality. 

10 Also, by performing image processing on an image 

obtained by synthesizing an image (natural image) obtained by 
photographing and a template, a synthesized image with a high 
picture quality can be obtained. 

In the case where a synthesized image that a user 

15 obtained with an application program is printed in a photo studio 

at user's request, an image-processing parameter for image 
processing is computed from the synthesized image, and image 
processing is performed on the synthesized image . However, since 
such a synthesized image contains both a natural image and a 

20 computer graphic (CG) image such as clip art contained ina template, 

the original color and brightness of the CG image will vary if 
image processing is performed on the synthesized image by the 
computed image-processing parameter. Also, because a CG image 
frequently contains primary colors compared to a natural image, 

25 the picture quality of a natural image will be degraded if image 

processing is perf ormedby an image-processing parameter computed 
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from a synthesized image containing a CG image. 

SUMMARY OF THE INVENTION 
The present invention has been made in view of the 
circumstances described above. Accordingly, it is an object of 
the present invention to provide an image processing method and 
image processor which are capable of performing suitable image 
processing on an image synthesized from a natural image and a 
CG image . Another object of the invention is to provide a program 
for causing a computer to execute the image processing method. 

To achieve the above-described objects, there is 
provided a method of performing image processing on an image 
synthesized from a natural image and a computer graphic (CG) 
image. The method comprises (a) a step of separating the 
synthesized image into a natural-image region and a CG-image 
region; (b) a step of computing an image-processing parameter 
for the image processing, based on the natural-image region; 
(c) a step of acquiring an intermediate image by performing the 
image processing on the synthesized image, based on the 
image-processing parameter; and (d) a step of acquiring a 
processed image by synthesizing the natural-image region 
contained in the intermediate image and the CG-image region 
contained in the synthesized image. 

In the image processing method of the present invention, 
a boundary portion between the natural-image region and CG-image 
region contained in the synthesized image may be blurred and 
then the CG-image region in the synthesized image and the 



natural-image region in the intermediate image may be 
synthesized. 

In accordance with the present invention, there is 
provided an image processor for performing image processing on 
5 an image synthesized from a natural image and a computer graphic 

(CG) image. The image processor comprises (1) separation means 
for separating the synthesized image into a natural-image region 
and a CG-image region; (2) parameter computation means for 
computing an image-processingparameter for the image processing, 

10 based on the natural-image region; (3) processing means for 

acquiring an intermediate image by performing the image 
processing on the synthesized image, based on the 
image-processingparameter; and (4) synthesis means for acquiring 
a processed image by synthesizing the natural-image region 

15 contained in the intermediate image and the CG-image region 

contained in the synthesized image. 

In the image processor of the present invention, the 
synthesis means may blur a boundary portion between the 
natural-image region and CG-image region contained in the 

20 synthesized image and then synthesize the CG-image region in 

the synthesized image and the natural-image region in the 
intermediate image . 

Note that the image processing method of the present 
invention may be provided as a program that is carried out by 

25 a computer. The program may be provided on a computer readable 

recording medium. 
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According to the present invention, a synthesized image 
is separated into a natural-image region and a CG-image region, 
and based on the natural-image region, an image-processing 
parameter for image processing is computed. Next, an 
5 intermediate image is acquired by performing the image processing 

on the synthesized image, based on the image-processing parameter . 
And a processed image is acquired by synthesizing the 
natural-image region contained in the intermediate image and 
the CG-image region contained in the unprocessed synthesized 

10 image. Since the image-processing parameter is computed based 

on the natural-image region contained in the synthesized image, 
suitable image processing is performed on a natural- image 
contained in the intermediate image. Therefore, a natural image 
contained in the processed image can have a high picture quality. 

15 On the other hand, the original color and brightness 

of the CG image contained in the intermediate image have varied 
due to image processing, but the processed image is obtained 
by synthesizing the unprocessed CG-image region in the 
synthesized image and the processed natural-image region in the 

20 intermediate image . Therefore, in the natural-image region, the 

natural image is able to have a high picture quality, and in 
the CG-image region, the processed image is able to maintain 
the picture quality of the original design. 

Also, according to the present invention, a boundary 

25 portion between the natural-image region and CG-image region 

contained in the synthesized image is blurred and then the CG-image 
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region in the synthesized image and the natural-image region 
in the intermediate image are synthesized. Therefore, there is 
no possibility that the boundary portion between a natural image 
and a CG image contained in the processed image will look unnatural . 
5 This enables the processed image to have a natural impression. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The present invention will be described in further 
detail with reference to the accompanying drawings wherein: 
FIG. 1 is an schematic block diagram showing an image 
10 processor constructed in accordance with a preferred embodiment 

of the present invention; 

FIG. 2 is a diagram used to explain how a natural-image 
region is specified; 

FIG. 3 is a diagram used to explain how a natural-image 
15 region and a CG-image region are separated; 

FIG. 4 is a diagram used to explain how natural-image 
regions are separated from CG-image regions; 

FIG. 5 is a diagram used for explaining how a maximum 
rectangular region is set; 
20 FIG. 6 is a diagram for explaining how a mask portion 

with a value of 255 is reduced after a filtering process; 

FIG. 7 is a diagram for explaining how a mask portion 
with a value of 255 is prevented from being reduced after a 
filtering process; and 
25 FIG. 8 is a flowchart showing the essential steps that 

are performed in the preferred embodiment of the present 
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invention . 

DESCRIPTION OF THE PREFERRED EMBODIMENT 
Referring now to Fig. 1, there is shown an image 
processor 1 constructed in accordance with a preferred embodiment 
5 of the present invention. As shown in the figure, the image 

processor 1 is equipped with read-out means 11, separation means 
12, setting means 13, image processing means 14 , synthesis means 
15, a monitor screen 17 for performing various displays, and 
input means 16 consisting of a keyboard and a mouse . The read-out 

10 means 11 reads out synthesized image data GO from a storage medium 

such as a CD-R, a DVD-R, a hard disk, a memory card, etc. The 
synthesized image data GO represents a synthesized image (also 
denoted by reference character GO) , which consists of a natural 
image, and a computer graphic (CG) image stored on a storage 

15 medium. The separation means 12 separates the synthesized image 

GO into the region of a natural image and the region of a CG 
image. The setting means 13 computes an image-processing 
parameter PO for performing image processing on the synthesized 
image data GO, as will be described later. The image processing 

20 means 14 acquires intermediate image data Gl (Gl also denotes 

an intermediate image) by performing image processing on the 
synthesized image data GO, based on the image-processing 
parameter PO . The synthesis means 15 acquires final image data 
G2 (G2 also denotes a final image) by synthesizing the region 

25 of a natural image contained in the intermediate image Gl, and 

the region of a CG image contained in the unprocessed synthesized 
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image GO . 

The read-out means 11 consists of a media drive, a card 
reader, etc., depending on the type of storage medium that is 
used. 

The separation means 12 first displays synthesized 
image data GO on the monitor screen 17, and then receives the 
specification of a natural-image region contained in the 
synthesized image GO, performed by a user. Fig. 2 shows how the 
natural -image region is specified . As illustrated in the figure, 
if synthesized image GO, which consists of a circular 
natural-image region Nl and a CG-image region CI other than the 
natural-image region Nl, is displayed on the monitor screen 17, 
the user specifies a rectangular region Rl containing the 
natural-image region Nl with the input means 16, as shown by 
a broken line in Fig. 2. In this way, the synthesized image GO 
is separated into a region Rl and a region R2 (shaded portion) 
other than the region Rl . 

The separation means 12 extracts a pixel having the 
same color as the color of pixels contained in the region R2, 
from all pixels within the region Rl . Next, the separation means 
12 judges that pixels having the same color as the color of pixels 
contained in the region R2, among all pixels within the region 
Rl, are CG-image pixels, and also judges the remaining pixels 
to be natural-image pixels. 

As illustrated in Fig. 3, the separation means 12 
separates the region Rl into a region Rll corresponding to a 
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natural image and a region R12 corresponding to a CG image, by 
extracting pixels that have the same color as the color of pixels 
contained in the region R2, from all pixels within the region 
Rl . In this manner, the synthesized image GO is separated into 
5 the region Rll corresponding to a natural image and the region 

corresponding to a CG image (hereinafter referred to as R3) . 
The region R3 consists of regions R2 andR12 . If region separation 
is accurately performed, then the region Rll will correspond 
to the natural-image region Nl and the region R3 will correspond 

10 to CG"image regions R2 and R12 . 

As illustrated in Fig. 4, there are cases where the 
region Rll corresponding to a natural image within the region 
Rl contains a small region LI consisting of a pixel having the 
same color as the color of pixels contained in the region R2 . 

15 Conversely, there are cases where the region R12 corresponding 

to a CG image within the region Rl contains small regions L2 
consisting of a pixel not having the same color as the color 
of pixels contained in the region R2 . In such a case, the size 
of the small regions LI, L2 is compared with a predetermined 

20 threshold value. When they are greater than or equal to the 

threshold value, it is judged that the small region LI is a CG 
image contained in a natural image and that the small region 
L2 is a natural image contained in a CG image. In this way, the 
natural-image region Rll and CG- image region R12 are determined. 

25 On the other hand, when the size of the small region 

LI or L2 is less than the threshold value, it is judged to be 
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noise. In this case, the small region LI within the region Rll 
is contained in the region Rll corresponding to a natural image. 
Conversely, the small region L2 within the region R12 is contained 
in the region R12 corresponding to a CG image. 

Note that the separation of the natural-image region 
Rll and CG-image region R12 may be displayed on the monitor screen 
17 so it can be corrected with the input means 16. 

As shown in Fig. 5, the setting means 13 sets a maximum 
rectangular region KO that is inscribed in the circular region 
Rll corresponding to a natural image extracted by the separation 
means 12. And based on the synthesized image data GO within the 
set maximum rectangular region KO, the setting means 13 computes 
an image-processing parameter PO for performing image processing, 
such as a density correction process, a color correction process, 
a gradation correction process, etc., on the synthesized image 
data GO . 

The image processing means 14 acquires intermediate 
image data Gl by performing image processing on the synthesized 
image data GO, based on the image-processing parameter PO computed 
in the setting means 13. 

The synthesis means 15 generates mask image data, and 
performs a filtering process on the mask image data, using a 
low-pass filter. The mask image data represents a mask image 
that causes the value of the region R3 corresponding to a CG 
image to be 255 (synthesized image data GO in the preferred 
embodiment is represented by 8 bits) and the value of the region 



Rll corresponding to a natural image to be 0. 

Fig. 6 depicts how a filtering process is performed 
in the vicinity of the boundary between the region R3 and region 
Rll by a low-pass filter . In the vicinity of the boundary between 
the regions R3 and Rll, if a filtering process is performed on 
the mask image data by a low-pass filter, the values of signals 
near the boundary vary smoothly as shown by a broken line in 
Fig. 6. 

However, in the mask image after the filtering process, 
a mask portion with a value of 255 is reduced within the region 
R3 corresponding to a CG image . As a result, the boundary portion 
of a CG image will blur when synthesizing a region corresponding 
to a natural image contained in the intermediate image Gl and 
a region corresponding to a CG image contained in the synthesized 
image GO, as will be described later. 

Because of this, the synthesis means 15, as depicted 
in Fig. 7, widens a region where a value in a mask image has 
255, in accordance with the filter size of a low-pass filter, 
and then acquires filtered mask image data by performing a 
filtering process on the mask image data with a low-pass filter. 
In the case of a low-pass filter having a filter size of 5 X 
5, if a region where a value in a mask image is 255 is widened 
by the amount corresponding to 2 pixels, filtered mask image 
data can be obtained without reducing a mask portion that has 
a value of 255 after a filtering process. 

The synthesis means 15 acquires processed image data 



(final image data) G2 that represents a processed image (final 
image) , by synthesizing the synthesized image data GO and 
intermediate image data Gl, based on the filtered mask image 
data. The synthesis is performed by the following Eq. 1: 

5 

C2(x.y) = ^^.G0(..3.) + 251^^.Cl(x,^) (1) 

in which (x, y) is the position of a pixel on an image represented 
by synthesized image data GO, Gl, or G2, and M is the filtered 

10 mask image data. 

Now, a description will be given of operation of the 
preferred embodiment. Fig. 8 shows the essential steps that are 
performed in the preferred embodiment. Initially, the read-out 
means 11 reads out synthesized image data GO from a storage medium 

15 (step SI) . Then, the separation means 12 separates the 

synthesized image data GO into a natural-image region Rll and 
a CG-image region R3 (step S2) . 

Next, based on the synthesized image data GO within 
the region Rll corresponding to a natural image, the setting 

20 means 13 computes an image-processing parameter PO (step S3) . 

And the image processing means 14 acquires intermediate image 
data Gl by performing image processing on the synthesized image 
data GO, based on the image-processing parameter PO (step S4) . 

Next, the synthesis means 15 acquires processed image 

25 data (final image data) G2 representing a processed imagq (final 
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image) G2 by synthesizing the synthesized image data GO and 
intermediate image data Gl, based on filtered mask image data, 
as indicated in the above-described Eq. 1 (step S5) . The 
processed image data G2 is printed out by a printer. 
5 Thus, in the preferred embodiment of the present 

invention, a processed region N2 corresponding to a natural image 
and an unprocessed region C2 corresponding to a CG image are 
synthesized into a processed image (final image) G2 . Since the 
image-processing parameter PO is computed based on the 

10 synthesized image data GO within the region Rll corresponding 

to the natural-image region Nl contained in the synthesized image 
GO, suitable image processing is performed on a natural image 
contained in the intermediate image Gl . Therefore, a natural 
image contained in the intermediate image Gl can have a high 

15 picture quality. 

On the other hand, the original color and brightness 
of a CG image contained in the intermediate image Gl have varied 
due to the above-described image processing, but the processed 
image (final image) G2 is obtained by synthesizing a region C2 

20 corresponding to the CG image CI contained in the unprocessed 

image GO, and the natural-image region N2 contained in the 
intermediate image Gl . Therefore, in the natural-image region, 
the natural image can have a high picture quality, and in the 
CG-image region, the processed image G2 has maintained the picture 

25 quality of the original design. 

In addition, the region C2 corresponding to a CG image 
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is obtained by performing a mask process on the synthesized image 
GO by filtered mask image data, so the circumference of the region 
C2 corresponding to a CG image blurs. Therefore, there is no 
possibility that the boundary portion between the natural image 
5 andCG image contained in the processed image G2 will look unnatural . 

This enables the processed image G2 to have a natural impression. 

In the preferred embodiment of the present invention, 
although the maximum rectangular region KO is set within the 
region Rll, and the image-processing parameter PO is computed 
10 based on the synthesized image data GO within the maximum 

rectangular region KO, the image-processing parameter PO may 
be computed based on the synthesized image data GO within the 
region Rll. 

Also, in the preferred embodiment of the present 
15 invention, although the synthesized image GO is separated into 

the region Rll corresponding to a natural image and the region 
R3 corresponding to a CG image, users may confirm the region 
Rll and region R3 by displaying them on the monitor screen 17 
after separation. In this case, it is preferable to be able to 
20 reset the separated region Rll and region R3 manually with the 

input means 16. 

The image processing method of the present invention 
may be provided as a program that causes a computer to execute 
the image processingmethod . The programmay be provided recorded 
25 on a computer readable recording medium. A skilled artisan would 

know that computer readable recording media are not limited to 
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any specific type of storage device and may be any type of device, 
including but not limited to: CD's, floppy disks, RAM's, ROM's, 
hard disks, magnetic tapes, and internet downloads, in which 
computer instructions can be stored and/or transmitted. 
5 Transmission of computer code through a network or through 

wireless transmission means is also within the scope of this 
invention. Additionally, computer code/instructions include, 
but are not limited to: source, object and executable code, and 
may be in any language, including higher level languages , assembly 
10 language, and machine language. 

While the present invention has been described with 
reference to the preferred embodiment thereof, the invention 
is not to be limited to the details given herein, but may be 
modified within the scope of the invention hereinafter claimed. 
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